The present invention relates to improvements in the processing of digitised x-ray images (particularly of the breast—termed mammograms or mammographic images), and more particularly to the enhancement of such images to assist clinicians in making accurate diagnoses based on them.
X-ray imaging is used as a basis for many medical techniques and, in particular, mammography continues to be the examination of choice for early detection of breast cancer in post-menopausal women and is the basis for national screening programmes.
Early detection of breast cancer greatly improves mortality rates, perhaps by as much as 25%. For this reason, mammographic examinations are nowadays performed on about 25 million women annually in the EC (of which, about 3 million are in the UK), at a cost of about 3 Bn US dollars per year. This huge cost and the poor accuracy in diagnosis (8-25% of cancers are missed and 70-80% of open surgical biopsies turn out to be benign), have led to increased interest in applying computer-aided techniques. Further, pressure for a reduction in the screening interval and for routine adoption of two-view screening (cranio-caudal and 45 degree medio-lateral) would entail a large increase in the number of mammograms to be analysed in the screening programme and this makes the development of reliable and robust computer techniques vital.
As an application of image processing, mammographic images pose a tough challenge because they have poor signal-to-noise ratio. This is largely because the images exhibit complex textures, and because there is a compromise between radiation dose and image quality. Worse, abnormalities appear as quite subtle, irregular, often non-local differences in intensity and the images are inevitably cluttered due to superimposition in the image of features separated in the breast. Further, the background varies greatly between different breasts, and there is relatively weak control of the imaging acquisition.
Unfortunately, while there have been proposals for the application of image processing to mammography, the vast majority have been of limited scope and incorporate only general non-mammography-specific image processing considerations. This involves great dangers. Image smoothing using such standard techniques may make lesions easier to locate, but can remove calcifications and spiculations which can be signs of cancer. Edge sharpening using standard techniques may appear to improve an image, but malignant lesions typically have fuzzy edges whereas benign ones tend to have sharp edges, so this edge sharpening process can actually transform an image of a malignant lesion into one that appears to a radiologist to be benign.
Calcifications present particular interest and problems. Localised cancers in the ducts or lobules of the breast are often are associated with secretions that thicken or become necrotic. These are called calcifications, or microcalcifications if they are smaller. Clusters of microcalcifications, which appear as small bright white objects in a mammogram, can be one of the earliest signs of breast cancer. Thus the identification of calcifications is a major goal of screening programmes, though benign calcifications are common (for example they often occur in blood vessels), and so the distinction between ductal and vascular microcalcifications needs to be made if the number of false positives is to be sufficiently low. This is not a problem for radiologists interpreting mammograms against the background of their knowledge of breast anatomy. It is, however, a challenge for image analysis programs. Further, dust and dirt entering the imaging system can create artifacts that mimic the appearance of microcalcifications and thus can be a cause of false positives both for radiologists and automated imaging systems.
Further problems are caused because the imaging process itself introduces a number of variables which affect the image and these will be explained below.
In the accompanying drawings FIG. 1 shows a schematic representation of the components of a conventional screen-film mammographic system. When a mammogram is performed, a beam of x-ray photons 1 from an x-ray tube 2 powered by a generator 4 is directed towards a breast 3 compressed between compression plates 17. This beam 3 is filtered by filter 5 to remove low energy photons and collimated by collimator 7 to the area of interest. The beam has a spectrum of energies that is characteristic of the tube voltage and, in particular, the material of the anode 9 but the spectrum is independent of the woman being scanned and the view taken. The intensity of the beam exiting the breast is related to the thickness and type of tissue in the breast. The x-ray photons leaving the breast normally have to pass through an anti-scatter grid 11 before they reach a phosphorous intensifying screen 13. If an x-ray photon is absorbed in the screen 13, light photons are emitted by the phosphor and these light photons expose a film 15 which is processed to produce a mammogram. The exposure to the breast is stopped once an automatic exposure control 19, positioned under a section of the breast, has received a set exposure. To generate a digital image the x-ray film is typically digitised using a laser scanner system or CCD and light box (not illustrated).
The intensity of radiation incident on the breast in such a system varies spatially for several reasons. The most significant is the “anode heel effect”. An x-ray tube produces x-rays by firing an electron beam at an anode. As the electron beam penetrates the anode the electrons are absorbed at varying depths and the x-ray photons that are produced have to travel through varying thicknesses of anode material before leaving the anode. This leads to varying attenuation of the emergent x-ray beam thus giving spatial variations in the incident x-ray spectrum, this is termed the anode heel effect and is quite substantial. Another source of spatial variation is due to the diverging nature of the beam. This means that the further away from the source the more spread out the x-ray beam is. However, this effect is small given that the distance from the source to the breast is large relative to the breast size. In visual assessment of mammograms the clinician mostly considers local variations in intensity, so the smooth change caused by the anode heel effect is not too troublesome. However, it does cause a problem for automated systems.
Two further effects of the image forming process which affect the image are scatter and extra-focal radiation. Considering scatter first, the x-ray radiation passes through the breast as shown in FIG. 2 and what is known as the primary beam passes through in a straight line from the anode to the intensifying screen. However, some x-ray photons are scattered in the breast and arrive at the screen from unexpected directions. The anti-scatter grid, which typically consists of a series of angled lead strips 21 separated by paper and aligned with the primary beam, removes many but not all of these scattered photons. Thus some scatter will reach the screen and be recorded on the film. Scatter can be estimated using the techniques described in “Computing the scatter component of mammographic images”, by Ralph Highnam, Michael Brady and Basil Shepstone published in IEEE Med. Imaging, 1994, 13, pp 301-313. This allows the calculation of the primary energy imparted to the screen to be improved by the subtraction of the energy due to scatter. Extra-focal radiation refers to radiation which comes around the edge of the collimator 7 as shown by numeral 1A in FIG. 3, and can constitute up to 15% of the total, some of which is scattered and reaches the film/screen. Simple techniques for measuring the extra-focal radiation are known, for instance from the paper by Highnam, Brady and Shepstone mentioned above.
The processes of intensification by the intensifying screen introduces blur or glare into the recorded image because the absorption of an x-ray photon at a point site 13A in the screen results in the approximately isotropic emission of light as shown in FIG. 4 which results in blurring of the image recorded on the film.
Further, the relationship between the density of the image on the film and the energy imparted to the intensifying screen is not linear and changes with film processing conditions. Again, this may not affect visual assessment which is based on local variations, but would affect automated analysis, especially if the non linearity were not explicitly taken into account.
Finally, the process of digitizing the film introduces digitizer blur into the digital representation of the image.
It will be appreciated, therefore that the enhancement of x-ray mammograms and automated recognition and differentiation of features in mammograms is a very difficult problem.
It has been proposed that the conversion of a digitised mammogram into a particular representation, termed the hint representation, is capable of improving the enhancement and analysis of such mammograms. This was described in “A representation for mammographic image processing” by Ralph Highnam, Michael Brady and Basil Shepstone published in Medical Image Analysis; 1996, vol. 1, no. 1, pp 1-18 and, since the present invention is concerned with improvements to it, will be briefly explained below.
The intensity of a mammogram at a given pixel (x, y) indicates the amount of attenuation (absorption and scattering) of x-rays in the pencil of breast tissue vertically above (x, y) on the film.
Ideally, one might hope to be able to produce a quantitative three-dimensional representation of the breast with each voxel labelled with a tissue type, such as: glandular, fibrous, cancerous, fat, calcium. Given the x-ray attenuation within a voxel it is certainly possible to classify fat since it has relatively low linear attenuation coefficients. It is also possible to classify likely occurrence of calcium, which is practically radio-opaque. However, the remaining breast tissues are those that comprise anatomically significant events in breast disease, such as cysts, malignant masses, fibroadenomas, and they are difficult to resolve from x-ray attenuation measurements alone. In the hint representation these remaining tissues are classified as “interesting tissue”. Further, there is actually very little calcium so for practical purposes it can be ignored.
Unfortunately, a further problem arises because of the projective nature of mammographic imaging: the three-dimensional information is lost. In light of this, the only information that is available describes the tissue within a cone of the breast, where the cone has as its base the area of a pixel and as its apex the x-ray source. After appropriate correction the x-ray beam within this cone can be considered as a pencil beam. Thus in the hint representation (with calcium ignored) there are basically only two tissue classes of fat and interesting tissue to consider, and the thicknesses of the interesting tissue (hint cm) and fat (hfat cm) which together must necessarily add up to the total breast thickness H (i.e. H=hint+hfat) are used as quantitative breast measurements.
In practice hint is computed from a mammographic image using data related to system calibration and image calibration. The x-ray tube output spectrum is assumed to be relatively stable but the anode heel effect is corrected for. The mammographic imaging process has several parts which might vary from day to day. In order to effect meaningful image analysis by computer, it is necessary to know these variations in order to make the images conform to a standard. To achieve this requires calibration data. The film-screen response, film processor and film digitizer are calibrated by collecting the following data:
1. A step wedge film: A film is produced with a stepped wedge made of lucite placed along the back of the film and a lucite block placed over the automatic exposure control. This film allows us to calibrate the film-screen system and film-processing so that energy imparted to the intensifying screen can be related to film density.
2. A “blank” film: A film is taken with a short time of exposure with no object (breast) present. The exposure has to be short so that the film does not saturate. An exposure of 0.04 seconds, at 100 mA and 28 kV for example produces a film that has film densities that vary between 1.8 and 2.6 (despite looking black). This film provides information about the spatial variations of the incident radiation intensity.
3. The digitized image of step wedge film: The film density on each step of the wedge is measured so that once digitized, the relationship between pixel value in the digital image and film density in the corresponding area of the film is known.
As well as calibrating the system components, data specific to each mammographic examination is needed. In particular:                The tube voltage (Vtube kV);        The tube current (Itube mA);        The time of exposure (ts s);        The breast thickness (H cm).        
Most of this information is readily available but measuring the breast thickness H is currently awkward since the radiographer has to measure it using a ruler; though newer machines are incorporating automatic measurement of breast thickness.
Given a mammographic image, the thicknesses of interesting and fatty tissue between the x-ray source and each pixel can be found by considering the energy imparted to the intensifying screen at each pixel which is obtained from the pixel values in the image using the calibration data. Let Epse(x, y) be the energy imparted to the screen in the area corresponding to the pixel (x, y). Epse(x, y) contains both scatter and primary components. The primary component Ep(x, y) is determined by subtracting a scatter estimate from the total energy imparted as mentioned above.
Now for a pixel with hint cm of interesting tissue and hfat cm of fatty tissue above the corresponding area of the intensifying screen, the total attenuation at any energy E is expected to be:
                                                                        h                ⁢                                                                  ⁢                                  μ                  ⁡                                      (                                          E                      ,                      x                      ,                      y                                        )                                                              =                                                                                          h                      int                                        ⁡                                          (                                              x                        ,                        y                                            )                                                        ⁢                                                            μ                      int                                        ⁡                                          (                      E                      )                                                                      +                                                                            h                      fat                                        ⁡                                          (                                              x                        ,                        y                                            )                                                        ⁢                                                            μ                      fat                                        ⁡                                          (                      E                      )                                                                                                                                                              =                                                                                                    h                        int                                            ⁡                                              (                                                  x                          ,                          y                                                )                                                              ⁢                                          (                                                                                                    μ                            int                                                    ⁡                                                      (                            E                            )                                                                          -                                                                              μ                            fat                                                    ⁡                                                      (                            E                            )                                                                                              )                                                        +                                      H                    ⁢                                                                                  ⁢                                                                  μ                        fat                                            ⁡                                              (                        E                        )                                                                                                        ,                                                          (        1        )            where the substitution hfat(x, y)=H−hint(x, y) is made.
In this case, the energy expected to be imparted to the intensifying screen by the primary photons is:
                                                        E              p                        ⁡                          (                              x                ,                y                            )                                =                                    ϕ              ⁡                              (                                                      V                    tube                                    ,                  x                  ,                  y                                )                                      ⁢                          A              p                        ⁢                          t              s                        ⁢                                          ∫                0                                  V                  tube                                            ⁢                                                                    N                    0                    rel                                    ⁡                                      (                    E                    )                                                  ⁢                                  ES                  ⁡                                      (                    E                    )                                                  ⁢                                  G                  ⁡                                      (                    E                    )                                                  ×                                  ⅇ                                                            -                                                                        μ                          plate                                                ⁡                                                  (                          E                          )                                                                                      ⁢                                          h                      plate                                                                      ⁢                                  ⅇ                                                            -                      h                                        ⁢                                                                                  ⁢                                          μ                      ⁡                                              (                                                  E                          ,                          x                          ,                          y                                                )                                                                                            ⁢                                                                  ⁢                                  ⅆ                  E                                                                    ,                            (        2        )            where φ is the photon flux for an x-ray tube voltage of Vtube, this varies across the image due to the anode heel effect; Ap is the pixel area; ts is the time of exposure; N0rel(E) is the relative number of photons at energy E; S(E) is the absorption ratio of the screen to primary photons of energy E, G(E) is the transmission ratio of the grid for primary photons of energy E; μiuc(E) is the linear attenuation coefficient of the (typically) lucite (compression plate) at energy E; and hplate is the thickness of the compression plate, all of which are known from the calibration or image conditions.
Note that after substituting Equation (1) into Equation (2) the only unknown is hint(x, y). This can be found by equating the primary energy found in the practical case (i.e. measured from the image) with the theoretical value (i.e. the expected value calculated above) and solving the resulting nonlinear equation to determine hint(x, y).
This process of converting the image into the hint representation can be visualised as converting the original image so that the fat has risen to float on top of the interesting tissue surface, then the fat is peeled off leaving the representation hint(x, y). Informally, this representation can be viewed as a surface and clinically significant effects such as masses appear as features on this surface, eg. small hills, as seen for example in FIG. 5. Note that this is fundamentally different from regarding the intensity image as a surface, since the hint representation is a quantitative measure of anatomical tissue which is distributed through vertical pencils of the breast. The importance of hint stems from the fact that it factors out the imaging parameters particular to the examination to yield a representation of the intrinsic anatomy that is ultimately what is relevant for diagnosis.